This invention relates generally to multimedia signal processing, and more specifically to adaptive coding, transmission and efficient display of digital content of multimedia.
Many international standards have been established for the communication of multimedia data, especially video and audio data, over a variety of communications networks. These coding and transmission standards and many existing solutions for coding and transmitting of multimedia arose largely in the context of a broadcast (or fixed media such as DVD) model of multimedia distribution. In a broadcast model of multimedia distribution, pre-compressed multimedia is broadcast (or stored in digital storage devices) to all users simultaneously, and all receivers (e.g., multimedia decoding devices) decode the same multimedia signal to the best of their abilities.
To allow multiple vendors to interoperate in multimedia coding and distribution, it was necessary to standardize only the syntax of an encoded bitstream of multimedia, e.g., by standardizing multimedia decoders, and leave the actual encoding to content providers. Thus, for the most part, the existing multimedia coding standards do not answer the question of how to optimize an entire communications channel, right up to human viewing of the processed multimedia; instead, most elements of the communication channel are addressed individually.
More and more digital multimedia content, e.g., digital video and audio, is now being delivered over Internet Protocol (IP) networks such as the Internet. The model of the effective communications channel (e.g., for home entertainment) is evolving rapidly in the marketplace, favoring direct and individualized streams for each receiver (e.g., web-based, streaming services) over the previous broadcast model. Even when the data is pre-compressed, there is now the ability to store various versions of the same content at different bit rates, allowing real-time stream switching to adjust to channel conditions (an approach generally referred to as “simulcast”). While TV still has a schedule, more and more digital content is now available in pre-stored format, created real-time or on-demand. The rapid development of processing and distribution of digital content of multimedia calls for more efficient solutions.
Moreover, there has been a massive proliferation of digital consumer devices on which the digital content of multimedia is consumed, from giant TV monitors all the way down to handsets. The needs and capabilities of the emerging consumer devices, coupled with the associated evolving user viewing conditions, all vary tremendously. In this situation, the previous one-size-fits-all approach for digital content processing and distribution is ineffective. Thus, the challenges of existing solutions to efficient multimedia processing and distribution problems and applications call for an efficient and robust system-level solution that meets the challenges.